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BASIC  OPTIMAL  ESTIMATION  AND  CONTROL  PROBLEMS 


IN  HILBERT  SPACE 

De  Santist  R.M.,  Saekst  R. , Tung®  L.J. 


0.  ABSTRACT 


The  recently  developed  mathemat ical  framework  of  Hilbert 
resolution  space  valued  random  processes  is  used  to  formulate 
and  solve  an  abstract  quadratic  optimization  problem.  By  parti- 
cularizing the  description  of  the  operators  appearing  in  the 
statement  and  solution  formula  of  this  problem  one  rediscovers 
and  generalizes  most  of  the  classical  estimation  and  control 
theory  problem  statements  and  results.  These  results  include, 
among  others,  the  Wiener  smoothing  prediction  filter,  the  Kalman  ' 
regulator,  the  Kalman- Bucy  filter,  the  stochastic  control  separation 
principle  and  the  more  recent  Youla-Jabr-Bongiorno  optimal  servo 
problem  solution. 
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BASIC  OPTIMAL  ESTIMATION  AND  CONTROL  PROBLEMS 
IN  HILBERT  SPACE* 


1 . INTRODUCTION 

Most  of  the  classical  [Wi  l]  and  modern  [Yo  l] , Wiener- 
Kolmogoroff  linear  least  square  optimization  methodology  is 
based  upon  frequency  domain  and  analytic  function  theory  proce- 
dures. As  a consequence,  most  of  its  applications  traditionally 
focus  on  the  realm  of  stationary,  lumped  parameters  and  infinite 
time  interval  systems  [Ne  l] , [Ch  l] . As  Bode  and  Shannon  [Bo  l] 
observed,  however,  the  computational  steps  involved  in  these 
procedures  have  a direct  physical  interpretation  in  terms  of 
causality  related  operations  and  random  signal  representations 
whose  meaningfulness  does  not  depend  on  any  one  of  these  special 
properties.  Thus,  it  is  perfectly  natural  to  look  for  extensions 
of  the  Wiener-Kolmogoroff  methodology  so  as  to  make  it  applicable 
to  systems  of  a more  general  type.  More  specifically,  the  following 
questions  are  of  interest:  can  the  Wiener-Kolmogoroff  methodology 
generalize  to  a form  which  is  independent  of  analytic  function  theory? 
can  this  potential  generalization  encompass  multivariate  cases  and 
related  frequency  domain  results  which  utilize  the  algebraic  Riccati 
equation?  can  nonstationary  systems,  finite  time  interval,  and  in- 
finite dimensional  state  space  problems  be  solved  using  the  genera- 
lized solution? 


# By  De  Santis,  R.M.,  Saeks,  R.,  and  Tung,  J.L. 


1 


2 


The  objective  of  the  present  paper  is  to  show  that,  on  the 
basis  of  recent  development*;  about  causality  [De  l] , [De  4] , 

[Sa  l] , [Sa  2] , state  [Sa  6] , [Sch  l] , [St  l]  and  stochastic 
signals  representation  [Kai  l] , [Kai  2] , [Sa  3] , [Ba  2] , the 
answer  to  the  above  questions  is  an  affirmative  one.  It  is 
indeed  now  possible  to  develop  a generalized  Wiener-Kolmogoroff 
methodology  which  while  encompassing  classical  optimal  estimation 
and  control  results  is  also  applicable  to  nonstationary,  finite 
time  interval,  and  distributed  parameters  systems.  The  advantages 
of  such  a generalization  are  multiple:  a better  perspective  of  the 
relations  among  a number  of  different  problems  and  techniques;  a 
clarification  of  the  role  played  by  causality,  state  and  related 
system  theoretic  concepts;  a wideT  Tange  of  application  of  the 
available  results. 

We  do  this  by  combining  the  classical  Wiener-Kolmogoroff 
ideas  and  techniques  [Wi]  with  the  novel  framework  of  abstract 
Hilbert  resolution  spaces  [Go  2 J . We  start  by  formulating  and 
solving  two  simple  optimization  problems,  (statements  PI  and  P2, 
Theorems  1 and  2) . Subsequently  we  show  that  by  appropriately 
specializing  the  description  of  the  operators  appearing  in  the 
statements  and  related  solution  formulas  of  these  problems,  one  can 
rediscover  and  generalize  the  solutions  of  most  of  the  optimal  esti- 
mation and  control  problem  of  interest,  namely:  the  Wiener  smoothing 
and  prediction  filter,  (Theorem  3},  the  Youla-Jabr-Bongiomo  optimal 
servo  compensator,  (Theorem  4) , the  Porter  basic  optimization 
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formula,  (Theorem  5),  the  Kalman  optimal  regulator  and  the  principle 
of  optimality,  (Theorem  6),  the  Kalman-Bucy  filter,  (Theorem  7),  and 
the  separation  principle,  (Theorem  8). 

The  development  inherits  ujch  from  the  previous  work  on  the 
-abject.  The  questions  under  consideration  were  first  formulated 
and  studied  by  Porter  [Po  l] . The  mathematical  setting  is  based  on 
the  Nonself-adjoint  Volerra  operators  techniques  developed  by 
Gohberg  and  Krein  [Go  l] , [Go  2]  and  on  the  more  recent  Hilbert  space 
valued  random  processes  framework  proposed  by  Balakrishnan  [Ba  l] . 

In  regard  to  the  results,  the  solution  of  our  basic  optimization 
problem  uses  ideas  and  techniques  from  Porter  [Po  l] , Bode  and  Shannon 
[Bo  l] , Kailath  [Kai  l],  Balakrishnan  [Ba  l] , and  Youla  [Yo  l].  A 
significant  role  in  the  proofs  of  the  state  estimation  and  control 
results  has  been  played  by  the  abstract  state  space  realization  work 
by  Steinberger,  Schumitzky  and  Silverman  [st  l]. 


\ MATHEMATICAL  BACKGROUND 

The  reader  will  be  assumed  to  be  familiar  with  those  standard 
notions  which  are  usually  associated  with  the  concepts  of  Banach  and 
Hilbert  space  [Po  2,  chp  l] , [Ba  2,  chp  l].  In  addition  to  these  we 
will  also  use  the  concepts  of  Hilbert  resolution  space  which  are  dis- 
cussed in  [Go  1,  chp  l] , [Sa  2],  plus  the  concepts  of  Hilbert  space 
valued  random  variables  [Ba  2,  chp  6],  [Sa  3].  These  latter  concepts 
will  be  briefly  reviewed  in  the  following  paragraphs. 

A bounded  linear  operator  P on  a Hilbert  space  H is  called  an 
orthoprojector  if  for  all  pairs  x,  yeH  one  has  that  <Px,  y>  “ <x,  Py> 
and  P(Px)  “ Px.  Given  a linearly  ordered  set  v with  minimum  and 
maximum  elements  tQ  and  ta  , we  will  say  that  a family  of  orthoprojec- 
tors, R “ {P1:  tev},  is  a resolution  of  the  identity  if  it  enjoys  the 

following  two  properties: 

t ts  . 

i)  P °H  ” 0,  P °°H  - H and  Ph  3 P*H  whenever 

k > l 

ii)  if  {P1}  is  a sequence  of  orthoprojectors  in  R and  there 
exists  an  orthoprojector  P such  that  {P'Sc}  Px  for  each 
xeH,  then  PeR. 

A Hilbert  space  H equipped  with  a resolution  of  the  identity 
R ” {P*:  tev}  is  called  a Hilbert  resolution  space  and  is  denoted  by 
the  symbol  [h,  P*] . Given  an  orthoprojector  PteR,  the  orthoprojector 
I-Pt  is  often  denoted  by  the  symbol  Pt- 
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We  will  use  the  triple  (H,  I,*)  to  denote  a probability  space 
of  Hilbert  space  valued  random  processes  with  Hilbert  space,  H,  family 
of  Borel  sets,  Z,  and  probability  measure,^.  Recall  that  [Ba  2,  chp  6], 
with  each  random  process,  pe(H,  E,  P),  with  a finite  first  and  second 
moment,  one  finds  associated  an  element,  mpeH,  (the  mean  value  of  p) 
and  an  operator,  Qp:  H -*■  H,  (the  covariance  of  p) , such  that: 

E(x,  p)  - (x,  mp),  VxeH 

and  E(x,  p-mp)  (y,  p-mp)  - (x,  Q^) , Vx,  yeH 

where  the  symbol  "E  f(p)"  denotes  the  expected  value  of  the  scalar 
valued  random  variable  f(p)  with  respect  to  the  probability  space 
underlying  p.  Similarly,  with  each  pair  p,  ire  (H,  E,f*)  one  finds 
associated  an  operator,  Qpif,  (the  cross  covariance')  with  the  property 
that : 

E(x,  p-mp)  (y,  ir-mp  - (x,  Qp7fy) , Vx,  yeH. 

In  the  sequel  we  will  not  explicitely  use  the  Z family  of  Borel  sets 
or  the  specific  nature  of  the  probability  measure  (F.  Thus,  for  more 
background  about  these  concepts  we  simply  refer  the  reader  to 
[Ba  2,  chp  2]. 

The  mean  values  and  the  covariance  and  cross  covariance  operators, 
however,  will  be  used  extensively.  While  the  theory  associated  with 
these  concepts  is  again  well  documented  in  [Ba  2,  chp  6] , for  our  pur- 
poses it  will  be  helpful  to  list  a few  of  their  most  important  properties 
namely: 


Ci)  “ m,  * m.  1 
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(ii)  |mp|  < E C 1 o | >; 


(iii)  “ Lmp  for  any  bounded  linear  operator  L; 

* 

(iv)  Q(Lp)  ^Tir^  “ LQpir  T for  bounded  linear  operators  L and  T; 

M v - % * v * «,„  * V 

<vi>  v - V 

I l2 

(vii)  E { I p ] } is  finite  if  and  only  if  Q is  nuclear  in  which 

. ,2 

case  E { |p|  } - tr(Qp); 

2 

(viii)  If  E { | p j > is  finite,  then  is  Hilbert-Schmidt  for 
all  random  processes  ir  with  a finite  second  moment. 

Note  that  given  an  operator,  T:  H H,  the  trace  of  T is  defined  by: 

tr(T)  - E (Te.,  e.) 

where  {e^}  is  any  orthonormal  basis  in  H.  A useful  property  of  the 
trace  is  that  for  T^,  T2:  H -*•  H one  has 

tr(T1T2)  - trt^Tj). 

The  random  processes  p and  ire  (H,  E,  will  be  said  to  be  sta- 
tistically independent  if  Qpff  “ 0.  In  the  course  of  our  development 
we  will  use  the  fact  that  linear  transformations  of  independent  random 
processes  are  independent,  plus  the  fact  that  if  p and  ir  are  indepen- 
dent then  Qp+ir  — Qp  + Unless  otherwise  specified  we  will  only 

consider  zero  mean  valued  random  processes. 


3.  CAUSALITY  AND  STATE  CONCEPTS 


The  concepts  of  causality  and  state  will  play  a fundamental 
role  in  the  development  of  the  following  sections.  While  a full 
treatment  of  this  subject  can  be  found  in  a number  of  references, 
(among  others  [De  l] , [Sa  2],  [Sa  6]  , [Sch  l]  , and  [st  l] ) , it  is 
helpful  to  review  some  of  the  basic  definitions  and  results. 


3.1  Basic  Definitions  and  Properties 

An  operator  T:  [h,  Pl]  -*•  [H,  P*]  is  said  to  be  causal  if 

Ptx  — P*y  implies  P*Tx  “ pS’y.  T is  strictly  causal  if  it  is  causal 

and  for  any  given  e > 0 one  can  find  a partition  {tQ  — Eq,  .... 

E “ t } ev  such  that 
n 00 

sup  | A.TA. | < e 
i 11 

where  A.  — P P . T is  called  anticausal  (strictly  anticausal)  if 

. 1 5i-l  ~~ 

T is  causal  (strictly  causal).  T is  memoryless  if  it  is  simultaneously 
causal  and  anticausal.  The  validity  of  the  following  results  is  easily 
verifiable. 


Lemma  1:  The  following  statements  are  equivalent:  T is  causal; 

P*x  - 0 implies  P*!*  - 0,  TPt  - PtTPt;  P*T  - P^TP*;  T*  is  anticausal. 

Leans  2:  Suppose  that  T^  is  causal  and  is  strictly  causal.  Then 
TjT2  and  TjTj  are  strictly  causal. 

Our  definition  of  strict  causality  even  though  most  convenient  from 
a technical  point  of  view  is  in  general  more  restrictive  than  what  it  will 
be  really  needed.  Indeed,  in  most  of  our  results  the  requirement  that 

sup  | A .TA . | < c could  be  replaced  by  sup  |A.TA.x|  < e,  where  the  partition 
i i 
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is  chosen  in  correspondence  with  a given  (x,  e)  pair.  A.  notable 
exception  to  this  is  given  by  the  following  lemma  whose  validity 
requires  the  restrictive  form  of  our  definition. 

Lemma  3:  Suppose  that  T is  a strictly  causal  operator.  Then  I ♦ T 

is  invertable  with  (I  + T)"1  * I «*  K,  where  K is  strictly  causal  and 

such  that  K - E (-1)"  Tn. 
n 

Given  a strictly  causal,  (strictly  anticausal),  T:  [h,  P*]  -►  [h,  P*], 
a state  realization,  (costate  realization),  of  T is  given  by  a triplet 
( fT(t) , Hj.(t) , tj.(t)  ) such  that  for  each  tev  one  has  that 

Hj.(t)  is  a Hilbert  space* 

*T(t):  PtH^Hr(t)  , (PtH*Hj.(t)), 

CT(t):  Hj.(t)-H»tH  , (Hj.(t)-H>tH), 

and,  for  each  ueH,  Cp(t)  fT(t)u  — P^TPSi,  (—  PlTP  u) . The  term  state 
realization  is  suggested  by  the  fact  that  for  every  tev  one  has  that 

PtTu  - Cj.(t)  x(t)  + PtTPtu 

where  x(t)  — V^(t)u.  Thus,  the  element  x(t),  the  state  of  T at  t, 
possesses  all  the  information  which  is  necessary  to  determine  the 
influence  of  the  past  values  of  the  input  over  the  future  values  of 
the  output.  In  [Sch  l]  it  is  shown  that  every  strictly  causal  operator 
admits  a state  realization. 

Given  a strictly  causal,  (strictly  anticausal),  T:  [H,  P1]  -*•  [h,  P*], 
with  state  realization,  (costate  realization),  (YT(t),  Hp(t),  £T(t)  ),  the 
pair  (kj.,  gT)  is  called  a state  trajectory  realization,  (costate  trajectory 
realization) , if  T “ gT  kj.  and 
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Itj.:  [h,  Pt]  -*■  Xj,,  kp  strictly  causal,  (strictly  anticausal); 

g?:  Xj,  -►  [H,  P*],  gT  memoryless,  and 

Xj.:  [L  [v,  Hp] , P*],  where  K,.  - Span  Hj.ft). 

tev 

Here  the  term  state  trajectory  is  justified  by  the  fact  that  the  element 
x — kj.u,  the  state  trajectory  associated  with  u,  can  be  viewed  as  a 
mapping,  x(»):  v ■*  Hpft),  where  x(t)  — f^ftju  represents  the  state  of  T 
at  t.  The  trajectory  space  Xj.  is  built  from  v and  Hj.  by  following  the 
procedure  outlined  in  [Ba  2,  section  3.5];  in  doing  this  the  linearly 
ordered  set  v is  tacitly  assumed  to  be  a measurable  subset  of  the  real 
line.  For  an  exhaustive  discussion  about  this  concept  the  reader  is 
addressed  to  [St  l].  Among  other  things,  this  reference  proves  that 
every  strictly  causal  T has  a state  trajectory  realization. 

In  working  with  the  above  causality  and  state  concepts  it  will  be 
useful  to  make  use  of  the  alphabetic  code  A — {A,  C,  M,  A,  C,  Co,  St} 
which  is  defined  as  follows 


A — anticausal 
C — causal 
M — memoryless 
£ — strictly  anticausal 


C — strictly  causal 
Co  — costate 
St  — state 


A first  application  of  this  code  is  given  by  the  following 


principle  of  causal  duality. 
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Lemma  4:  Let  a statement  or  equality  by  phrased  using  relations 
involving  concepts  associated  with  the  alphabet  A “ {A,  C,  M,  A,  C, 
St,  Co)  and  the  family  of  projection  operators  {P1}  and  (Pt).  Then 
the  statement  or  equality  remains  valid  if  the  following  interchange 
in  symbols  occurs: 

P*  Pt,  P -►  Pt,  C -*•  A , Ah-  C, 

C -*■  A , A -*•  C , St  -►  Co,  Co  -*■  St. 


3.2  Causality  Additive  and  Multiplicative  Decompositions 


An  operator  T is  said  to  admit  a causality  additive  decomposition 
[De  1],  [Sa  2]  if  T can  be  viewed  as  given  by  the  sum  of  a causal  plus 
an  anticausal  component;  T is  said  to  admit  a canonical  causality 
decomposition  if  it  can  be  represented  in  terms  of  the  sum  of  a strictly 
causal,  a strictly  anticausal  and  a memoryless  component.  The  following 
lemma  establishes  the  uniqueness  of  such  a decomposition  plus  a useful 


property  relating  the  causality  decomposition  of  T with  those  of  FT  and 
TF,  where  F is  a memoryless  system. 

Lemma  5:  Suppose  that  the  operator  T has  a canonical  causality  (addi- 
tive) decomposition,  T — T^  + Tg  + TM-  Then  the  elements  To,  oe  {A,  C,  M)  are 
uniquely  defined.  Moreover,  if  F is  memoryless  and  T — T^  + T^  + T^,  then 


Tj  — TF  and  T2 


T.  “ T - + T - 
i iA  iC 


where  T.  “T  F and  T_ 
la  a 2a 


— FT  are  such  that 

+ TiM*  i-1’  2 
■ FT  , ae  {A,  £,  M). 

(X 


r 


i 


I ° 


6 


r> 


t 


n 


In  view  of  lemma  5 , in  general  it  makes  sense  to  talk  about 
the  causal  (strictly  causal,  anticausal,  ...)  component  of  an 
operator  T:  [H,  Pt]  [h,  P*]  . For  example,  given  T — T-  + Tg  * T^, 
the  causal  component  of  T,  usually  denoted  by  Tc  or  [t]c  is  given  by 
Tg  + Tm>  The  causal  component  of  an  operator  T,  (as  well  as  the 
other  strictly  causal,  memoryless,  ...  etc.,  components)  does  not 
always  exist.  General  necessary  and  sufficient  existence  conditions 
plus  a number  of  interesting  causality  decomposition  related  properties 
can  be  found  in  [De  l] , [Sa  2]  and  [De  3] . One  of  these  properties 
will  be  of  a particular  interest  later  on  in  the  development.  To 
characterize  it,  consider  a partition  n “ {5  “ tQ,  Sj,  ...»  ?n  “ t^}  ev, 

and  associate  with  it  the  operator  valued  function 

n i 

*(0)  - 2 A.TP* 

i-1  1 

where  — P^*  - P^"*;  P*  “ P^*.  If  [t]^  is  well  defined  then  given 
any  e > 0 one  can  find  a partition  such  that  for  every  £1  o £l£  one 
has  that 

I [T]c  - *00  I < e 

where  the  norm  can  be  intended  either  in  the  uniform  or  in  the  strong 
operator  topology.  This  property  can  be  expressed  using  the  following 
symbols 

{l  T P1}  (M)/dPTPS  > [T]  . 

In  the  particular  case  of  Hilbert-Schmidt  operators  a canonical 


causality  decomposition  is  always  well  defined.  Moreover  the  components 
of  this  decomposition  are  mutually  orthogonal.  All  this  is  established 
by  the  following  lemma. 
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Lemma  6:  Causality  and  Hilbert- Schmidt  operators 

i)  the  class  of  causal,  (anticausal,  memoryless),  Hilbert-Schmidt, 
operators  is  a Hilbert  space; 

ii)  in  the  Hilbert-Schmidt  inner  product  topology  one  has 
< Ta,  Tg  > — 0 whenever  a t 8e  {A,  C,  M}; 

iii)  if  T is  a Hilbert-Schmidt  operator  then 
A C ‘M 

where  the  T^,  oe  {A,  C,  M},  are  well  defined  Hilbert-Schmidt 

operators  such  that 
min 

llT  " T0II  “ t NT  - TaH 

a 

where  is  any  other  a operator  and  the  symbol  ( | 1 | denotes 
the  Hilbert-Schmidt  norm. 

Equal  in  importance  to  the  concept  of  a causality  additive  decom- 
position is  that  of  causality  multiplicative  decomposition  also 
referred  to  as  causal  factorization.  An  operator  Q:  [H,  Pt]  ■*  [h,  Pt] 
admits  a causal  factorization  if  Q can  be  represented  in  terms  of  the 
cascade  composition  of  a causal  (anticausal)  operator  followed  by  an 
anticausal  (causal)  component.  In  the  sequel  we  will  mainly  be  con- 

* 

cerned  with  operators  Q which  are  positive  Hermitian  (<Qx,  x>  >0,  Q — Q ) 
and  with  causal  factorizations  of  the  type  considered  in  the  following 
lemma  [Sa  3]. 


' * 
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Lemma  7:  If  Q:  [H,  Pt]  ■+•  [h,  P*]  is  positive  Hermitian  then 

i)  there  exists  a causal  and  causally  Left  invertable  one-to-one 
0:  [H,  P*]  -*•  [H,  P*]  such  that 
Q “ on* 

ii)  there  exists  a causal  and  causally  right  invertable  one-to-one 
A:  [H,  P*]  ■»  [h,  P*]  such  that 
Q “ A*A 


X 


3.3  Causal  Decomposition  and  State  Realization 

The  following  two  lemmas  generalize  two  well  known  linear  time 
invariant  results  ([Bar  l],  [Ma  l]).  They  unveil  some  key  relations 
concerning  a special  multiplicative  causality  decomposition,  the 
causal  part  of  a related  operator  and  the  concept  of  state  trajectory. 
It  is  precisely  by  virtue  of  these  relations  that  we  will  be  able  to 
elucidate  the  connections  between  a plant  causality  and  state  structure 
on  the  one  hand,  and  the  structure  of  the  optimal  filter  and/or 
compensator  on  the  other.  The  proof  of  the  first  part  of  these  lemmas 
can  be  found  in  [St  l] . The  second  part  follows  by  involving  the  prin- 
ciple of  causal  duality.  In  stating  these  lemmas,  the  symbols  F and  B 
will  denote  two  memoryless  operators,  while  G will  represent  a strictly 
causal  system.  We  will  suppose  that  (GB,  F)  is  a state  trajectory 

realization  of  FGB  and  (G*F*,  B*)  is  a costate  trajectory  realization 

* • • 
of  B G F . 
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Lemma  8: 


i)  Suppose  that  there  exists  a strictly  causal  Vj  such  that 


I * B G F F6B  *“  (I  + Vp  (I  + Vx) 


then  (t) 


[(i  + vj)"1  b*gVfgb]c  - vr 


ii)  If  there  exists  a strictly  causal  V2  such  that 
I + FGBB*G*F*  “ (I  + V2)  (I  + V*) 


[fgbbVf*  (i  + v2)_1]c  - v2. 


Lemma  9: 


i)  Suppose  that  there  exists  a strictly  causal  such  that 

i * b*gVfgb  - (I  + V*)  (I  + vx) 

then  there  exists  a well  defined  memoryless  such  that 
Vj  - MrGB. 

ii)  If  there  exists  a strictly  causal  V2  such  that 

i ♦ fgbbVf*  - (I  * V2)  (I  + V*) 

then  there  exists  a well  defined  memoryless  M such  that 

© 

* w * * 

V2  - MeG  F . 


The  next  lemma  provides  the  key  to  the  stochastic  control  prin- 
ciple of  separation  (section  6). 


(+)  Recall  that  the  symbol  [Tjr  denotes  the  causal  component  of  T. 
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Lemma  10;  Suppose  that  there  exists  strictly  causal  V1  and  V2  such 


that 


and 


I ♦ B G F FGB  - (I  + Vp  (I  + vp 


I + FGBB  F “ (I  + V2)  (I  + V2) 

Then  one  has  that 

[Cl  ♦ V*)'1  B*G*F*FGBB*G*F*  (I  + V*)"1^  - MRGMe 
where  and  Mg  are  memoryless  and  such  that 

1 

Proof:  From  the  strict  anticausality  of  G F (I  + V2)  one  has 

. * _ i * * * * * * 

[(I  + Vp  B G F FGBB  G F (I  + V2)  A]c  “ 


V,  - Mj^GB  and  V2  - FGMe< 


, r *_!***  , * * * * _1  , 

L [(I  * Vp  B G F FGB]C  B G F (I  + V2)  X]c 


Moreover,  from  lerana  8i) 

. *_!***  - 

[(I  + Vx)  B G F FGB]C  - Vj 

and,  from  lemma  8ii) 

[fgbbVf*  (i  + v2)"1]c  - v2 

Using  these  equalities  and  applying  lemma  9,  it  follows  that 

[(i  ♦ vj)'1  bVf*fgb]c  -mrgb 


and 


[FGBB#gV  (I  + V*)"1]c  - FGMe. 


This  leads  to 


where  F 
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r r.  * _1  * * * , * * * * _ 1 . 

[ [(I  + Vj)  1 B G F FGB]C  B G F Cl  + V2)  A]c  - 
Mj^F"1  [FGBB*G*F*  (I  + V*)-1]c  -MRGMe. 
has  been  used  to  denote  the  pseudo  inverse  of  F. 


( 


4.  TWO  FUNDAMENTAL  PROBLEMS 


In  the  course  of  the  development  it  will  be  shown  that  many  of 
the  quadratic  optimal  control  problems  considered  in  the  technical 
literature  can  be  viewed  as  special  cases  of  a "basic  stochastic 
optimal  control  problem";  a key  step  in  the  solution  of  this  problem 
will  in  turn  be  provided  by  the  solution  of  what  we  will  call  a 
"best  causal  approximant  problem".  With  this  in  mind  we  find  it 
convenient  to  proceed  as  follows.  We  start  by  stating  and  solving 
the  best  causal  approximant  problem.  Subsequently  we  discuss  the 
basic  stochastic  optimal  control  problem.  In  the  forth  coming  sections 
we  will  show  how  these  preliminary  results  allow  us  to  solve  a variety 
of  other  problems  of  interest. 

4.1  The  Best  Causal  Approximant  Problem 

The  best  causal  approximant  problem  can  be  stated  as  follows 
(see  Figure  1). 

Statement  PI:  The  best  causal  approximant  problem. 

Given  a not  necessarily  causal  II:  [H,  P1]  ■+•  [H,  P*] , and  a random 

process  ze  (H,  Z,  P),  with  covariance  operator  Q2>  determine  a causal 

4 such  that  for  any  other  causal  4 one  has  that 
o 

JC4q)  < J (4) 

where 

J(4)  -E  { | | (4  - n)  z 1 1 S - tr  {(4  - H)QZ  (4  - II)*}. 

The  following  theorem  gives  us  a sufficient  condition  for  the 
solution  of  the  best  causal  approximant  problem. 


17 


18 


Theorem  1:  A sufficient  condition  for  a causal  $ to  be  a solution 

o 

of  the  best  causal  approximant  problem  is  that 

!)»o  - [n»]c 

where  £1  is  causal  and  such  that 

q2  - an*. 

Proof:  Let  us  first  consider  the  case  where  Q — I,  that  is  the  case 
where  z is  a white  noise  random  process.  For  any  partition 
UQ  " tQ,  ....  cN  - tj  ev  one  has 

JO)  - e { | cn  - *)z|2} 

N i 

— S E { 1 A.  (IIP1  - *)z  + A.np.zl  } 
i-1  1 11 


wheTe  P1  — P^i  and  — PXP^  From  here,  taking  into  account  the 

statistical  independence  of  P1z  and  P^z,  it  follows 

N . 2 N 2 

JO)  - EE  { I A . (np-1  - *)z|  } + I E { I A.np . z I } 
i-1  i— 1 1 1 

N 

Recalling  that  by  a partition  refining  process  we  have  that  l A. IIP1 

i-1  1 

converges  to  [n]^,  it  follows 

JO)  -E  ( | C[n]c  - *)z|2}  + e { | (n  - [n]c)z|2} 

Since  the  second  term  of  this  sum  is  independent  from  9,  it  follows 
that  the  problem  of  minimizing  JO)  is  equivalent  to  that  of  mini- 
mizing the  first  term.  This  is  obviously  done  by  choosing  — [n]^. 

* 

Let  us  consider  the  more  general  case  where  Qz  — ^ I,  that  is 

the  case  where  z is  a colored  random  process.  This  case  can  be 
reduced  to  the  special  white  noise  case  by  regarding  z as  given  by 


MM 
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the  output  of  the  causal  filter  ft  driven  at  the  input  by  a white 
noise  random  process  o>.  Using  the  notations  II'  *•  lift  and  ft'  — ftft 
this  leads  to 

■ ,2 

JO)  — e { | cn  - ft)ftu>|  } 

- e (Jen*  - *')u[2} 

From  here  one  can  repeat  the  white  noise  argument  to  conclude  that 
J(ft)  is  indeed  minimized  by  ft'  — ftft  ~ [nn]^. 

Note  that  Theorem  1 does  not  imply  that  J(#q)  < « . If  J(ft) 
is  not  finite,  however,  no  other  ft  will  make  J(ft)  finite.  These 
difficulties  may  be  alleviated  by  assuming  that  certain  operators 
are  Hilbert-Schmidt.  Using  these  assumptions  Theorem  1 can  also  be 
proved  in  a more  direct  and  mathematical  more  transparent  fashion. 

As  this  alternative  fashion  puts  in  a better  perspective  the  role 
played  by  Hilbert-Schmidt  operators  and  their  special  properties  it 
is  in  order  to  introduce  the  following  alternative  proof. 

An  alternative  proof  of  Theorem  1:  Assume  that  ft  and  H are  Hilbert- 
Schmidt  and  observe  that 

j(*)  - e {|(*  - n)z|2}  - tr  {(ft  - n)Q,  (ft  - n)*} 

Z 

* 

From  Qz  — nft  , with  ft  causal,  and  denoting  by  {e^},  a complete 
orthonormal  set  in  H,  it  follows  that 

j(ft)  - tr  {(ft  - n)nn*  (ft  - n)*} 


- {Z  < (ft  - n)£lei,  (ft  - n)£lei  >} 

i 

- ll(ft  - n)n| |2 


where  the  symbol 


denotes  the  Hilbert-Schmidt  norm.  Observing 
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now  that  #8  is  causal,  and  recalling  that  the  causal  and  strictly 
anticausal  parts  of  a Hilbert-Schmidt  operator  are  well  defined  and 
mutually  orthogonal,  (lemma  6),  it  follows  that 

•>(♦)  - IN  - [no]cl|2  * II  [noljll2. 

2 

From  here,  observing  that  ||  | is  independent  of  9,  one  can 

conclude  that  a sufficient  condition  for  9 to  be  the  desired  solution 

o 

is 

V - [nn]c- 

The  following  corollary  considers  an  important  special  case  of 
the  best  causal  approximant  problem. 

Corollary  1:  Suppose  that  for  a given  P°eR  one  has  that 

Qz  - P0G  + P V°>  G*Po 

with  G causal.  Then  a sufficient  condition  for  9 to  be  a solution 

o 

of  the  best  causal  approximant  problem  is 

W - Kc]c- 

Proof : Observe  that  in  this  particular  case  one  has 

J(«)  " * J2(*2) 

with 

•VV  - lie*!  - “)  PoG||2 

and 

J2(*2)  - tr  {(«2  - n)  P0GP°QuP°G*P0  (*2  - n)*}. 


The  proof  can  then  be  completed  by  verifying  that  a sufficient  con- 
dition for  these  two  latter  functionals  to  be  simultaneously  minimi- 
zed is 

♦olG  - ‘o2°  - KG)c  • 

This  is  done  by  observing  that 

W " llpo  (*1P0G  ■ tnpoGJc)H2  + IIP0  [nP0G]cl|2  * II  [nP0G]j||2 
and 

J2(«2)  - tr  {P0  C.2P0G  - [IP0G]C)  P\P°G*P0  (*2  - Pon)* 

♦ tr  {P°np  GP°Q  P°G*P  H*P0}  . 

0 (1)  0 

The  last  equality  follows  from 

P0nP0GP°  - pQ  [hp0g]cp° 

and  the  fact  that 

tr  {P°nP  GP°Q  P°G*P  (4  - PI)*P  ) - 0 
tr  {PQ  (*2  - PoII)  PoGP0Qb)P0G*Pon*P°}  - 0 . 


4.2  The  Basic  Stochastic  Optimization  Problem 

8 

The  following  problem  is  inspired  from  the  "basic”  deterministic 
problem  studied  by  Porter  in  [Po  l].  In  the  spirit  of  this  reference 
% we  will  retain  the  "basic"  qualifier  because  as  we  shall  see  in  the 

later  sections,  this  problem  includes  as  a special  case  many  other 
fundamental  problems  such  as  the  Wiener  filter,  the  Kalman  optimal 


• regulator,  the  optimal  Kalman-Bucy  state  estimation  problem. 
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Statement  P2:  The  basic  stochastic  optimization  problem  (Figure  2). 
Given  four  not  necessarily  causal  systems  1^:  [h,  P*]  -*■  [h,  Pl] , 

i ™ 1,  2,  3,  4;  and  two  random  processes  a,  Be  (H,  I,  *);  determine 
a causal  controller,  Dq,  such  that  for  any  other  causal  0 one  has 

J(D0)  < 

where 

J(D)  - E { j ® 1 2 ♦ |r|2} 
and 

e — LjD  (Lja  ♦ B)  - I^a 
r - L4D  (Lja  + B) . 

The  next  theorem,  the  key  result  of  the  overall  development,  says 
that  under  some  appropriate  hypotheses  the  basic  optimization  problem 
is  equivalent  to  an  associated  best  causal  approximant  problem;  its 
solution  can  then  be  obtained  by  using  Theorem  1. 

Theorem  2:  Suppose  that  there  exists  causal  and  causally  invertable 
fl  and  A such  that 

« " Wl  * % * Ll^aB  * Vl  ‘ “* 

and 

LjLj  ♦ “A  A . 

Then  the  basic  optimization  problem  is  equivalent  to  a best  causal 
approximant  problem  with 

q2-q 

and 

n - A*-1  4 CW;  * Q-*. 


A solution  of  the  basic  optimization  problem  is  given  by 


Bo  ~A'1  tA*'ll3  (wl  * 


Proof:  Observe  that 

J (D)  -E  { | (LjDLj  - L2)o  + L3D0 1 2 + E { | LjDLjO  + L4DB|2} 

- tr  Xj  ♦ tr  X2 

where 

X1  " L3DLlW*L3  * Lj“JBD*L3  * L3DLAbD’L3  * L3DVL1D"L3 

* w,lIdX  * w*lI  * l4dlAbd'lI  * wi0*1! 

and 

X2  ”’L3DLlQaL2  " L2QaLl°  L3  * L2Qo8D  L3  “ L3DQ0oL2  + L2QoL2  * 


Observe  also  that 

tr  Xj  - tr  { CL3L3  ♦ L*L4)DQD*> 

where 

« ■ wi  * % * hu  * v-i  • 


Taking  into  account  that 

L3L3  * L4L4  " A’A 


it  follows  that  J(D)  can  be  rewritten  as 
J (D)  - tr  {ADQD*A*  + X2>  . 

From  here,  using  some  straightforward  algebraic  manipulations  plus  a com- 
pletion of  the  square  argument  similar  to  that  used  in  [Bo  l]  one  obtains 

J(D)  -J0(D)  + tr  {noQn*  ♦ L^L*} 
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where 

"o  ■ a''‘l3  <wi  * W5’1 

and 

Jo(D)  - tr  {(AD  - no)Q  (AD  - !Io)*}. 

* 

Observing  now  that  the  term  tr  {-nQQIIo  + is  independent  of  D, 

it  follows  that  the  problem  of  minimizing  J(D)  is  equivalent  to  that 
of  minimizing  Jq(D).  The  noncausal  solution  of  this  problem  is 
obviously  given  by  D “ A-1IIo.  The  causal  solution  corresponds  to  the 
solution  of  a best  causal  approximant  problem  with  * “ AD,  Qz  — Q,  and 
n ” nQ.  This  solution  can  then  be  obtained  by  applying  Theorem  1. 

The  following  corollary  is  interesting  in  that  it  provides  a useful 
relation  between  the  best  causal  approximant  problem  and  a special  ver- 
sion of  the  basic  optimization  problem.  This  result  will  be  applied 
in  the  study  of  the  Kalman  optimal  regulator  problem  (Theorem  5). 


Corollary  2:  If  a — 0,  Lj  ” 0 and  there  exists  a causal  and  causally 
invertable  A such  that 


L3L3  + L4L4 


* 

A A 


then  a sufficient  condition  for  DQ  to  be  a solution  of  the  basic  opti- 
mization problem  is  that 


ADo 


o 


where  is  a solution  of  the  best  causal  approximant  problem  with 

Q “ Q and  n - A*_1L*L-. 
z 3 2 
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Remark  1:  Using  lemma  7 one  can  see  that  the  invertibility  require- 
* * 

ments  on  + L^L^,  fl  and  A are  not  essential  to  the  validity  of 
Theorem  2,  nor  to  that  of  Corollary  2.  They  are  merely  used  to  sim- 
plify notations.  If  these  requirements  are  not  satisfied  one  would 
have  to  replace  A-1  with  A-R  (right  inverse)  and  A*"1  with  A*’L  (left 
inverse) . 


OPTIMAL  INPUT-OUTPUT  ESTIMATION  AND  CONTROL 


In  this  section  we  will  exploit  Theorems  1 and  2 to  obtain  the 
solution  of  three  fundamental  optimal  estimation  and  control  problems 
of  the  input-output  type.  For  a physical  motivation  of  these  problems 
and  related  classical  background  we  address  the  reader  to  [Bo  l] , 
fYo  l]  and  [Po  3]. 


5.1  The  Wiener  Filter  Problem 

The  following  statement  provides  a generalized  version  of  the 
Wiener  filter  problem. 


Statement  3:  The  Wiener  filter  problem  (Figure  3). 

Given  two  not  necessarily  causal  systems  Lj,  [H,  P1] , and  two 
random  processes  a,  Be  (H,  I,  ? ),  determine  a causal  filter  Dq  such 
that,  for  any  other  causal  D one  has  J(Dq)  < J(D),  where 

J (D)  “ E { J L2ot  - D (Lja  + B)|  }. 

The  following  theorem  gives  a sufficient  condition  for  the  solution 
of  the  generalized  Wiener  filter  problem. 

Theorem  3:  Suppose  that  the  operator 

' LlV3  + <*B  + LAb  * Wl 


is  invertable.  Then  the  noncausal  solution  of  the  Wiener  filter  problem 
is  given  by 
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A causal  solution  is  given  by 

Do  - tl2 

* 

where  n is  causal,  causally  invertable  and  such  that  Q — Oft  . 

The  proof  of  Theorem  3 follows  directly  from  Theorem  2 by 
recognizing  that  the  Wiener  filter  problem  can  be  viewed  as  a 
specialized  version  of  the  basic  optimization  problem  with 
Lj  — 1 and  L4  “ 0. 

Note  that  just  as  in  the  classical  case,  c[b°  l] , p.  424),  the 
solution  procedure  of  the  generalized  Wiener  filter  is  implemented 
in  terms  of  the  following  physical  steps: 

i)  calculate  a prewhitening  filter  which  reduces  the 
colored  noise  problem  to  a white  noise  problem;  let 
this  be  ft; 

ii)  calculate  the  optimal  noncausal  filter  associated 
with  the  white  noise  problem;  this  is 

U - cl2v!  * 

iii)  calculate  the  best  causal  approximant  of  n;  let  it  be 

Me1 

iv)  construct  the  optimal  causal  filter  in  terms  of  the 
cascade  composition  of  the  inverse  of  the  prewhitening 
filter  ft"*,  followed  by  [ll]^;  that  is  Dq  - Weft*1. 

Thus  our  abstract  Hilbert  space  generalization  of  the  Wiener  filter 
enjoys  the  same  engineering  interpretation  which  characterizes  the 
classical  transfer  function  development. 
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An  additional  point  of  interest  in  regard  to  Theorem  3 is  that 

by  setting  “ 0 and  L2“  1,  it  also  gives  an  immediate  solution  to 

the  following  optimal  estimation  problem  (Figure  4):  Given  a and  8, 

. ,2 

determine  a causal  D so  as  to  minimize  E ( D6  - a }.  This  latter 

o 1 1 

problem  was  formulated  and  studied  in  [Sa  3].  The  following  corollary 
rectifies  the  incorrect  solution  given  in  that  reference. 

Corollary  2:  If  0-  is  invertable  then  a sufficient  condition  for  D 

P O 

to  be  a solution  of  the  optimal  estimation  problem  is 

Do  - V1- 

* 

where  £1  is  causal,  causally  invertable  and  such  that  Q “ £lfl  . 

P 


5.2  The  Optimal  Servo  Problem 


We  will  now  consider  an  extended  version  of  the  optimal  servo 
problem  which  has  been  studied  among  others  by  Newton,  Gould  and 
Kaiser  [Ne  lj,  Chang  [Ch  l] , and  Youla  [Yo  l] . Our  formulation  and 
result  are  formally  identical  to  those  which  can  be  found  in  this 
latter  reference  (compare  our  Theorem  4 with  Corollary  1,  in  [Yo  1, 
part  II]).  A formal  statement  of  the  problem  is  as  follows. 


Statement  P4:  The  optimal  servo  problem  (Figure  5). 

Given  the  statistically  independent  random  processes,  d,  l,  m,  and 
ue  (H,  £ ,f),  and  the  causal  systems  F,  Fq,  L,  Lq,  T,  Tq,  and  Tg,  all 
mapping  [H,  P*]  -*■  [H,  PtJ . Determine  a causal  feedback  controller  Nq 
such  that 


i)  N is  causal; 
o 

ii)  (I  * FTNq)  admits  a causal  bounded  inverse; 


I 
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iii)  For  any  other  causal  controller  N satisfying  i)  and  ii) 
one  has 

J(Nq)<  J(N) 

where 

J(N)  - E {|u  - y|2  + x |y | 2 

y - TN  (I  + FTN)"1  [u  + L0*  - Fom  + (L  - FTQ)d]  + TQd 
r -TN(I*  FTN)'1  [u  + L l - F m + (L  - FT  )d] . 

The  following  lemma  establishes  the  equivalence  between  the  optimal 
servo  problem  and  the  basic  optimization. 

Lemma  11:  Suppose  that  Dq  is  a solution  of  the  basic  optimization  pro- 
blem with 

8"L  l ■ Fb  f u 1 (L-FT)d 
0 0 o 

a - u - T0d 


Then  a sufficient  condition  for  a causal  N to  be  a solution  of  the 

o 

Optimal  Servo  Problem  is  that  (I  ♦ FTNq)  admits  a causal  bounded 
inverse  and 

(I  - FTD0)N0  - Dq. 
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Proof : From  Figure  5 one  has 

y - TN  (I  + FTN)'1  [u  + LqA  - FQm  + (L  - FTQ)dj  + TQd 

r - TgN  (I  * FTN)"1  [u  ♦ L Ql  - FQm  + (L  - FTQ)d] 

Using  the  notation 


S-u*Li-Fi*  (L  - FTJd 
o o o 


a — u - T d 
o 


it  follows 


e-u-y-a-  TNq  (I  + FTNq)  6 


r - TsNq  (I  + FTNq)  B 


Letting  now  D “ (I  * N FT)_1N  we  have 

O 0 0 


N “ D (I  - FTD  ) 
oo  o 


-1 


hence 


N (I  + FTN  )_1  - D 
o'-  0J  o 


From  here  we  obtain 


J (N)  - E { | e| 2 + k2  | r | 2>  - E {|o  - TOo|2  + |kTsDQB|2}  - J(Dq) 

Suppose  now  that  DQ  is  a solution  of  the  basic  optimization  problem  and 
that  Nq  is  not  a solution  of  the  optimal  servo  problem.  Using  the 
above  equations  this  would  imply  that  there  exists  an  N such  that 

JCN)  < J(Nq)  - JCDq). 


& 


t 


1 

1 
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Setting  D ■*  N (I  + FTN)  * we  would  then  have 
J(D)  - J(N)  < J(Dq) 

which  is  a contradiction  to  the  hypothesis  that  Dq  is  a solution  of 
the  basic  optimization  problem. 

The  next  lemma  gives  a sufficient  condition  for  the  solution  of 
the  basic  optimization  problem  which  lemma  11  associates  with  the 
optimal  servo  problem.  Its  proof  follows  directly  from  Theorem  2. 

Lemma  12:  Suppose  that  there  exists  causal  and  causally  invertable 
A and  ft  such  that 

* 2 * * 

TT+kTT  -AA 
s s 

Wo  * Wo  + Qu  + (L  • "o^  CL  - no>* 

Then  the  basic  optimization  problem  associated  with  the  optimal  servo 
problem  has  the  following  solution 

■>„  - A 1 [A  * 

where 

^8  " <*u  * Vd  <L-  FV* 

The  next  theorem  illustrates  the  conditions  under  which  the  vali- 
dity of  the  servo  problem  formula  proposed  by  Youla,  Jabr  and  Bongiorno 
can  be  extended  from  the  rational  transfer  function  to  the  more  general 
Hilbert  space  setting  under  consideration.  The  proof  is  an  immediate 
consequence  of  lemmas  11  et  12. 


t 
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Theorem  4:  Let  the  hypothesis  of  lemma  12  be  satisfied  and  suppose 

that  (I  - TFD„)  1 admits  a causal  inverse, 
o 

Then  a solution  of  the  optimal  servo  problem  is  given  by 
N0  - Dc  (I  - FTDorl 

provided  that  (I  + FTNq)  admits  a causal  bounded  inverse  • 

Remark  2:  Observe  that  using  the  notation  W —DO  the  solution 
— — — oo 

proposed  by  the  above  theorem  can  be  rewritten  as  follows 

»„  -»„«>-  my-1  • 

This  solution  is  formally  identical  to  that  which  can  be  found  in 
[Yo  2].  In  view  of  this  parallelism  it  is  in  order  to  clarify  that 
our  approach  is  at  one  time  more  general  and  more  particularized  than 
Youla's.  More  general  in  that  it  treats  infinite  dimensional  and  time 
variant  systems;  more  particularized  in  that  it  does  not  allow  the 
open  loop  chain  to  be  unstable. 

5.3  The  Deterministic  Basic  Optimization  Problem 

The  reasoning  underlining  the  proof  of  our  solution  of  the  basic 
optimization  problem  can  also  be  applied  to  cases  where  the  invertibi- 
lity  hypothesis  of  that  result  are  not  satisfied.  We  will  show  this 
by  formulating  and  solving  a slightly  more  generalized  version  of  the 
basic  (deterministic)  optimization  problem  studied  by  Porter  [po  3] . 
The  formulation  goes  as  follows. 


■MW 
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Statement  P5:  Porter  basic  deterministic  optimization  problem 
(Figure  6). 

Given  three  causal  systems  Gi : [h,  Pt]  ->■  [h,  Pt] , i — 1,  2,  3,  and 
an  element  zeH  such  that  Pqz  — z;  for  some  P°  — (I  - PQ)eR;  determine 
a causal  controller,  Dq,  such  that  for  any  other  causal  D one  has 

J(Dq)  < J(D) 

where 

J (D)  “ |(G2  - GjD) z | 2 * |G3Dz|2. 

The  following  theorem  rediscovers  Porter's  main  result,  (Theorem  2, 
[Po  3])  using  the  basic  optimization  problem  approach. 

Theorem  5:  Suppose  that  there  exists  a causal  and  causally  invertable 
A such  that 

G1G1  + G3G3  " A A 

Then  a sufficient  condition  for  D to  be  a solution  of  the  deterministic 

o 

basic  optimization  problem  is 

V“A"lpo 

Proof:  Observe  that  Porter's  deterministic  basic  optimization  problem 
corresponds  to  the  special  case  of  our  (stochastic)  basic  optimization 
problem  characterized  by  -■  0 and  o “ 6 “*  z.  This  is  precisely  the 
case  considered  in  Corollary  2.  Applying  this  corollary  and  taking 
into  account  the  following  correspondence  of  notations  L2  ™ G2,  Lj  ■“  G^, 
and  “ G^,  it  follows  that  if  causal  is  a solution  of  the  best 

*_i  * 

causal  approximant  problem  with  II  - A G1G2*  then  a solution  of  the 
deterministic  basic  optimization  problem  is  given  by  DQ  — A~**o. 


34 


Observe  also  that,  given  Qz  — 00  , a sufficient  condition  for  a causal 
to  be  a solution  of  the  best  causal  approximant  problem  in  that 
— HO;  (this  follows  immediately  from  the  second  equation  in  the 
proof  of  Theorem  1).  Noting  that  in  our  case  Qz  — (♦,  z)z,  and  that 
ft  "■  * z is  such  that  Qz  ■■  flfl  , it  follows  that  a sufficient  con- 

dition  for  a causal  to  be  a solution  of  the  best  causal  approximant 
problem  is 

(•,  z)*oz  - (•,  z)A*-1G*G2z. 


In  addition  to  this  from  the  causality  of  4>q  and  the  hypothesis 
that  z — Pqz  one  has,  (lemma  1), 

4 P z — P $ z 
o o oo 


hence 


* 

$ z “PA  G.G,z 
o o 12 


and  therefore 

-1  -1  r *- 1 * , 

A \z-A  XPo  [A  1G1G2z], 


This  last  equation  implies  that  a sufficient  condition  for  a causal 
0Q  to  be  a solution  of  the  deterministic  optimization  problem  is  that 

D0Z  -A‘lpo  [A*_1G*G2z]. 


OPTIMAL  STATE  ESTIMATION  AND  CONTROL 


* 


X 


X 
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In  the  same  spirit  of  the  previous  section  we  will  proceed  to 
apply  Theorems  1 and  2 to  three  fundamental  problems  of  the  optimal 
state  estimation  and/or  control  type.  In  contrast  with  the  usual 
approach,  where  one  looks  directly  for  an  optimal  state  feedback 
strategy,  and  in  line  with  [St  l]  we  find  it  more  appropriate  to 
formulate  these  problems  as  open  loop  problems.  It  will  be  an  inte- 
resting consequence  of  our  results  that  the  open  loop  solution  can 
be  implemented  in  terms  of  a memoryless  state  feedback  configuration. 

The  physical  motivation  of  these  problems  and  related  applica- 
tions are  once  again  well  documented  in  the  classical  references 
[Ka  l],  [Ka  2],  and  [An  l] . 


6.1  The  Optimal  Regulator  Problem 

We  will  start  with  a generalized  version  of  the  optimal  regulator 
problem  originally  proposed  and  solved  by  Kalman  [Ka  l].  This  abstract 
formulation  and  its  subsequent  solution  can  be  viewed  as  a natural 
outgrowth  of  similar  results  by  Steinberger,  Schumitzky  and  Silverman 
[St  l];  as  such,  the  discussion  of  these  authors  about  motivations  and 
advantages  of  the  Hilbert  resolution  space  approach  to  the  problem 
apply  "verbatim''  to  the  present  context. 

Statement  P6:  The  optimal  regulator  problem  (Figure  7) . 

Given  two  statistically  independent  random  processes  u,  nefHj,  Ej, 
with  — I,  and  three  causal  systems 

B:  [Hj,  ?\]  - [H2.  P*] 

G:  [H2»  P\]  - [H2.  P\] 

F:  [H2,  P*]  - [H3.  P*] 


! 
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with  B and  F memoryless  and  G strictly  causal  such  that  (GB,  F)  is 
a state  trajectory  realization  of  FGB.  Determine  a causal  optimal 
open  loop  regulator  Dq:  [Hj,  P*]  -►  [H^,  P*]  such  that  for  any 
other  causal  D one  has 

J(Dq)  < J (D) 

where 

J(D)  “ E { jy | 2 + |u | 2 } 

with 

y — FGB  (oi  + P°ir  - u) 

u - DP  FGB  (u  + P°tt) 
o 

Our  solution  of  the  optimal  regulator  problem  is  formalized  by 
the  following  theorem. 

Theorem  6:  Principle  of  optimality  and  the  Kalman  regulator. 

Suppose  that  there  exists  a strictly  causal  such  that 

i + bVf*fgb  - (I  + vj)  (I  + vx) 

Then  a sufficient  condition  for  Dq  to  be  an  optimal  open  loop  regulator 
is 

V “ o * VM'X 

is  a memoryless  operator  such  that  Vj  “ 
control  provided  by  this  regulator  can  be  implemented  by  using  the 
following  memoryless  state  feedback  law 

x *■  GB  (w  + P°ir  - u) 

u ■ v- 


where 


M^GB.  The  optimal 
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Proof:  Using  the  following  correspondence  of  notations 


a “ S - PqFGB  (a>  + P°it) 


Lj  - 0 


L2-i 


L,  - FGBP 

J o 


L4  ""  1 


It  is  easily  recognized  that  the  optimal  regulator  problem  is  a 
special  case  of  the  basic  optimization  problem  (Figure  8).  Moreover, 
by  observing  that 


implies 


I + B G F FGB  - (I  + Vj)  (I  + Vx) 


I + P B*G*F*FGBP  - (I  + P V*)  (I  + V.P  ) 
O O 01  lo 


one  also  recognizes  that  this  special  case  is  precisely  the  one 
considered  in  Corollary  1. 

Applying  that  result  we  have  the  following  sufficient  condition 
for  a causal  Dq  to  be  the  optimal  regulator 

AD  “ (I  + V-P  )D  - * 
o loo  o 

where  9Q  is  a solution  of  the  best  causal  approximant  problem  with 
Q2  “ and  n given  by 

n - a \i2  - a + p^)  pqb  g f -p0(i  + v1)  ‘bgf 


1 


38 


To  characterize  such  a #Q,  observe  that  from  the  statistical  indepen- 
dence of  (i>  and  ir  one  has 

Qa  “ " PoFGB  % + PV°)G*F‘B‘Po 

From  here,  applying  Corollary  1,  one  finds  that  a sufficient  condition 
for  #o  to  be  a solution  of  the  best  causal  approx imant  problem  under 
consideration  is 

*oFGB  “ 

that  is 

*0FGB  - [(I  + P0Vp‘lpoB*G*F*PoFGB^C 

- PQ  [a  + v*)"1b*g*f*fgb]c. 

Noting  that  by  applying  lemma  8i)  one  has 

[npQFGB]c  - Pq  [Cl  + v*)'VgVfgb]c  - pqv1 

where,  Cfrom  lemma  8),  — M^GB  with  MR  a well  defined  memoryless 

operator.  It  follows 

[nP0FGB]c  - M„PoGB 

hence  a sufficient  condition  for  0 is 

o 

[BP0FGB]C  - «0FGB  - P^GB 

From  these  equations  one  obtains  that  a sufficient  condition  for 
Dq  to  be  an  optimal  regulator  is 

D0F  - (I  * - a * MjGB)-1^ 
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This  formula  implies  in  turn  that  the  optimal  control  law  given 
by  Dq,  u,  satisfies  the  following  equation 

u - (I  + Mj^GB)  -1MrPoGB  Cto0  + P°tt) 


hence 


u - MgGBu  - MrPqGB  Cu  + P 


From  here  one  obtains  the  desired  memoryless  state  feedback  strategy. 


namely 


u-V 


x “ PqGB  Cu  + P it  - u) 

The  physical  interpretation  of  this  strategy  is  illustrated  by  the 
block  diagram  in  Figure  9. 

It  is  interesting  to  observe  that  according  to  the  above  theorem 
the  solution  of  the  optimal  regulator  problem  can  be  implemented  either 
by  means  of  an  open  loop  configuration  or  by  a closed  loop  configuration 
using  state  feedback.  In  general  the  optimal  closed  loop  configuration 
is  characterized  by  a sensitivity  behavior  which  is  better  than  that  of 
the  open  loop  configuration.  To  illustrate,  observe  that  if  we  repre- 
sent  with  ycH2  t*ie  influence  of  a given  perturbation  over  the  state  of 
the  optimal  open  loop,  then  (I  + Mj^GBJ'^y  will  represent  the  influence 
of  that  same  perturbation  over  the  state  of  the  optimal  closed  loop 
system.  Using  the  notation  $ — (I  + Mj^GB)'1  it  follows  that  the  optimal 
closed  loop  system  will  have  a better  sensitivity  behavior  than  the 
optimal  open  loop  if  and  only  if  ||$||  < 1,  that  is  if  and  only  if 
I - $*$  > 0,  [Po  l].  Note  that 

!-$*$-  $*  - I>  $. 
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From  here,  using  the  fact  that  Vj  — M^GB  it  follows 

* 

c « w * i vjuj  j 


I - $*$  " (I  + V*)"1  {bVf*FGB}  (I  + V.)"1  > 0 


One  can  then  conclude  that  the  closed  loop  configuration  of  the 
optimal  regulator  does  indeed  offer  a better  sensitivity  behavior 
than  the  open  loop  solution. 

6.2  The  Optimal  State  Estimation  Problem 

The  optimal  state  estimation  problem  originally  proposed  by 
Kalman  [Ka  l] , and  Kalman  and  Bucy  [Ka  2]  is  in  our  present  context 
formulated  as  follows. 

Statement  P7:  The  optimal  state  estimation  problem  (Figure  10). 

Given  four  statistically  independent  random  processes  ue(H1#  Zj,  <Pp, 
id,  ire(H2,  Z2>  (?2),  ne(H3,  Z3>  <Pj)  with  — I and  — I;  and  three 
causal  systems 

B:  [Hj,  ?\]  - [H2,  P*] 

G:  [H2.  P21  - [H2.  P2] 

P:  [H2.  P2]  - [H3,  pJ] 

with  B and  F memoryless  and  G strictly  causal  such  that  (GB,  F)  and 
* * * 

(G  F , B ) are  state  and  constate  trajectory  realizations  of  FGB  and 
* * * 

B G F respectively.  Determine  a causal  filter  Dq  "D  j,  Dq2),  with 


°„1:  [»j.  I’ll  - [»2.  P2] 
Do2:  CH1'  Pll  * &V  P2] 
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such  that  for  any  other  causal  D — CDj,  D2)  one  has 
J(D0)  ? J (D) 

where 

JCD)  - E {|x  - x|2} 

with 

x *"  GB  (u  + a)  + P°ir) 

A 

x — D^z  + D2u 
and 

z - PQ  [FGB  (u  + a)  + P°ir)  + n]. 

The  following  theorem  is  to  the  effect  that  the  optimal  state 
estimation  problem  can  be  solved  via  a generalized  Kalman-Bucy  filter. 

Theorem  7:  Suppose  that  there  exists  a strictly  causal  V2  such  that 

I ♦ FGBB*G*F*  - Cl  + V2)  (I  * V*) 

Then  a solution  to  the  optimal  state  estimation  problem  is  given  by 
°0l  - Ci  * Foy*1 

Dq2  - CI  - GMe  (I  + FGMe)-1F)GB 

where  M#  is  a memoryless  operator  such  that  V2  — FGMg.  The  optimal 
state  estimation  can  be  implemented  by  medns  of  the  following  state 
feedback  model 

x — G [M  [z  - Fx]  - u ! 
z - FGB  (u  ♦ w ♦ P°ir)  + n 
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Proof : For  simplicity  we  will  treat  the  case  where  ir  — 0;  the  proof 
for  the  case  ir  f 0 is  obtained  by  applying  Corollary  1 and  by  using 
an  argument  identical  to  that  developed  in  the  proof  of  Theorem  5. 
Observe  that  in  this  case 

.2 


J CD)  — E { |Dj  (FGB  (U  + u>)  + n)  ♦ D2u  - GB  (u  - u) | } 


-E  { |DX  (FGBw  ♦ n)  - GBoi  - (D2  * (DjF  - l)GB)u|  > 


that  is,  (from  the  statistical  independence  of  u,  u>  and  n) , 


2 2 
J (D)  -E  { |Dj  (FGBw  + n)  - GBu|  > + E {|  (D2  + (DjF  - I)GB)u|  } 


It  follows 


J(D)  < min  J. (D. ) + min  J (D. , D-) 

" n n n O i L 


D1*D2 


where 


JjCDj)  “ E {|D1  (FGBu  + n)  - GBu|  } 


and 


JoCDl*  °2)  “E  1 1 C°2  + (DjF  - I)GB)u | >. 


Noting  that  the  functional  Jq(Dj,  D2)  is  minimized  by  any  (Dj,  D2) 
pair  such  that 


d2  - C1  - D^GB 


one  can  see  that  the  problem  of  minimizing  J(D^,  02)  is  equivalent  to 
that  of  minimizing  This  latter  problem  in  turn  is  equivalent 

to  a basic  optimization  problem  with  “ I and  L4  — 0.  This,  given 
the  hypothesis  of  the  theorem,  is  precisely  the  problem  considered  in 
Theorem  3.  Taking  into  account  that  ” I,  Qg  — I,  “ 0,  Qga  “ 0, 


Lj  “ FGB  and  L 2 ■*  GB,  and  applying  that  result  we  then  have 
Dq1  - [GBB*gV  (I  + V2)_1]c  (I  + V2)-1 

From  here,  using  the  memorylessness  of  F one  obtains 

FDoi  “ [FGBB*gV  Cl  ♦ v*)'1]c  (I  + v2)_1 

and  by  observing  that  (see  lemma  8) 

[fgbbYf*  (I  + v*)"1]c  - v2 

it  follows 

FDol  -v2  Cl  ‘Vj)-1. 

Applying  lemma  9 we  have  V.  - M*G*F*  and  therefore  V_  - FGM  . It 

» “ 2 e 

follows 

FDq1  - FGMe  (I  + FGMe)_1 

hence 

»ol  “ Cl  * FGMe)-‘ 
and 

Do2  " Cl  - GMe  (I  ♦ FGMe)_1F)GB. 

The  state  feedback  realization  of  the  optimal  state  estimation  filter 
is  obtained  by  first  recognizing  that  the  optimal  estimation  provided 
by  (D0j»  Dq2)  is  equivalent  to  that  given  by  the  block  diagram  in 
Figure  11.  Subsequently  one  verifies  that  this  block  diagram  is  equi- 
valent to  that  in  Figure  12. 
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6.3  The  Stochastic  Optimal  Control  Problem 

In  what  follows  we  revisit  the  optimal  regulator  problem  by 
considering  the  case  where  the  output  of  the  system  is  corrupted 
by  noise. 

Statement  P8:  The  optimal  regulator  problem  in  the  presence  of 
measurement  noise  (Figure  13) . 

Given  three  statistically  independent  random  processes,  u>,  ire(Hj,  Z^,  fj) 
and  ne(Hj,  Zj,  <?3) , with  — I,  and  Q ~ I;  and  three  causal  systems 

B:  [Hr  ?\]  + [H2,  P*] 

G:  [H2,  - [H2,  P*] 

F:  [H2,  P*]  - [H3,  P3] 

with  B and  F memoryless  and  G strictly  causal  such  that  (GB,  F)  and 
* * * 

(G  F , B ) are  admissable  state  and  costate  trajectory  realizations 

* * * 

of  FGB  and  B G F respectively.  Determine  a causal  regulator 
D0:  [h3»  P3]  ■+•  [Hj,  P*]  such  that  for  any  other  causal  D one  has 

J(D0)  < J(D) 

where 

J (D)  “E  {|u|2  + |y|2} 

with 

u — Dz 

z - PQ  [FGB  (<o  + P°ir)  + n] 
and 

y ■*  z - FGBu. 
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Proof:  Using  the  following  correspondence  of  notations 
a “ PqFGB  (a>  + P°ir) 

8 “ PqFGB  + P°ir)  + n 


one  can  once  again  see  that  the  problem  under  consideration  is 
equivalent  to  our  basic  optimization  problem.  For  simplicity  and 
without  any  loss  of  generality,  we  will  again  confine  ourselves  to 
the  case  ir  — 0;  in  Theorems  6 and  7,  the  proof  for  the  case 
it  ^ 0 would  require  a slightly  more  involved  argument  based  on 
Corollary  1 and  similar  to  that  developed  in  the  proof  of  Theorem  6) . 

We  will  then  proceed  by  applying  Theorem  2.  Note  that 

I + B*G*F*FGB  - (I  + V*)  a + Vj) 

and 

i + fgbbVf*  - a + v2)  Ci  + v*) 

imply 

: + pqbVf*fgbpo  - ci  + p^)  a + v1pq) 

and 

I * PqFGBB*gV  - CI  + V2Po)  CI  + PoVj) 


Using  these  relations  and  applying  Theorem  2 one  obtains 


Do  “ C1  + V^)"1  [Cl  + PoV*)_1PoB*G*F*PoFGG*F*Po  (I  + P^)"^  (I  + V 

In  addition  to  this,  by  virtue  of  lemma  9,  Vj  — and  ^2  ™ P®*e, 
and  from  lemma  10 

- * * * * _i, 

[Cl  + Vj)  IGF  FGBB  G F (I  + V2)  *]  - MRGMe< 


It  follows  that  the  optimal  regulator  is  given  by 
°0  “ P0  U + MRGB)_1MRGMe  (I  * FGMJ-1 


The  proof  that  the  optimal  control  law  can  be  implemented  in  terms 
of  a feedback  loop  consisting  of  the  optimal  state  estimator  followed 
by  the  optimal  regulator  is  easily  obtained  by  verifying,  by  inspec- 
tion, that  the  block  diagrams  in  Figures  14,  15  and  16  are  mutually 
equivalent. 

Remark  3:  The  statements  of  the  problems  and  the  associated 
Theorems  6,  7 and  8 can  be  made  slightly  more  general  by  assuming 
” Q and  — R with  Q and  R memoryless  and  positive  definite.  To 
illustrate  how  this  is  done,  observe  for  example  that  if  in  the  sys- 
tem in  Figure  7,  one  has  — Q ? I,  and  — R t I then  one  can 
consider  the  optimal  control  problem  for  the  equivalent  system  repre- 
sented in  Figure  17  where  Q-  “ I,  Q-  “ I and  B “ 8 ™ GQ*^, 

F — R~  ' F.  Note  that  B,  8 and  F enjoy  the  same  state  and  causality 
properties  of  the  original  B,  G and  F. 


CLOSURE 


Using  the  Hilbert  resolution  space  valued  random  processes 
framework  developed  in  [Sa  3]  plus  recent  causality  and  state 
results  [De  l],  [st  lj , a best  causal  approximant  and  a basic 
stochastic  optimization  problems  have  been  formulated  and  solved. 

These  solutions  provide  the  key  to  a better  understanding  and  a 

significant  generalization  of  a number  of  classical  estimation  and 

automatic  control  theory  results.  In  the  context  of  optimal 

problems  of  the  input-output  type,  they  yield  a generalized  version 

of  the  Wiener  filter  (compare  [Bo  l]  and  Theorem  3) , the  Youla- 

Jabr-Bongiorno  servo  problem  solution  procedure  (compare  Theorem  4 

with  [Yl,  part  II,  Corollary  l]),  and  Porter's  solution  to  th» 

deterministic  basic  optimal  control  (compare  [Po  1,  Theorem  3]  and 

our  Theorem  5).  In  regard  to  estimation  and  control  problems  of 

the  state  space  feedback  type,  they  lead  to  generalized  versions 

of  the  Kalman  optimal  regulator  (compare  [Ka  1,  Theorem  4]  with 

our  Theorem  6),  the  Kalman-Bucy  filter  ([Ka  2,  Theorem l]  should  be 

compared  with  our  Theorem  7) , and  the  stochastic  control  separation 

theorem  ([Kw  1,  Theorem  5.3,  p.  390]  should  be  compared  with  our  Theorem  8). 

These  generalizations,  to  the  effect  that  most  of  the  classical 
results  in  optimal  estimation  and  control  theory  are  now  extended 
to  multivariable,  distributed  parameters  and  time  variant  systems 
are  of  course  not  surprising.  Most  of  them  have,  in  fact,  in  one 
form  or  the  other  already  appeared  in  the  technical  literature.  In 
particular,  for  example,  a generalization  of  the  Kalman-Bucy  filter 
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has  been  given  by  Falb  [Fa  l] , using  the  integration  theory  for 
Banach  space  valued  functions  of  Dunford  and  Schwartz,  by  Kailath 
[Kai  2]  in  the  context  of  the  innovations  approach  circle  of  ideas, 
and  by  Balakrishnan  [Ba  3]  in  the  setting  of  infinitesimal  generators 
on  strongly  continuous  groups.  Similarly,  a deterministic  version  of 
the  principle  of  optimality  has  been  given  among  others  by  Porter  [Po  l] 
and  in  a yet  more  comprehensive  form  by  Steinberger,  Schumitzky  and 
Silverman  [st  l],  A generalized  version  of  the  Wiener  filter  has  been 
given  by  various  researchers  including  Kailath  [Kai  2],  and  Balakrishnan 
[Ba  l].  Finally,  a generalized  separation  principle  has  been  offered 
among  others,  by  Balakrishnan  [Ba  l],  Wohnam  [Wo  l],  and  Curtain  [Cu  l]. 

The  main  contribution  of  the  paper  is  in  showing  that  our  Hilbert 
resolution  space  approach  represents  a worth  while  addition  to  the 
available  methods:  it  allows  to  obtain  all  these  and  more  results  as 
particular  cases  of  one  central  result;  it  retains  the  same  physical 
and  intuitive  character  of  the  classical  developments  by  Bode  and  Shannon 
[Bo  lj  and  Kalman  [Ka  l];  it  elucidates  the  connections  between  the  plant 
structural  properties  with  respect  to  such  concepts  as  state,  strict 
causality  and  causality  decomposition  and  the  structural  properties  of 
the  optimal  controller  and/or  observer.  Last  but  not  least,  it  opens  up 
a number  of  new  interesting  questions  in  the  realm  of  causal  factorization, 
optimal  control  and  Riccati  differential  equations. 
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1.  The  best  causal  approximant  problem 

2.  The  basic  optimization  problem 

3.  The  Wiener  filter  problem 

4.  The  stochastic  estimation  problem 

5.  The  optimal  servo  problem 

6.  Porter's  basic  optimization  problem 

7.  The  optimal  regulator  problem 

8.  Establishing  the  equivalence  between  the  optimal  regulator 
and  the  basic  optimization  problems 

9.  Kalman  optimal  regulator 

10.  The  optimal  state  estimation  problem 

11.  The  optimal  state  estimator  given  by  the  Wiener  filter 

12.  The  Kalman- Bucy  filter 

13.  The  optimal  regulator  problem  in  the  noisy  case 

14.  The  optimal  open  loop  regulator  in  the  noisy  case 

15.  The  optimal  closed  loop  regulator  in  the  noisy  case 

16.  The  optimal  regulator  in  the  noisy  case  as  given  by  the 
separation  theorem 

17.  Equivalent  white  noise  problem  associated  with  an  original 
coulored  noise  problem 
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FIGURE  2:  The  basic  optimization  problem: 

given  Lj,  a and  8 determine  a causal  D so  as. 

2 2 ‘ 
to  minimize  E {|r|  + |e|  }. 


X 


FIGURE  3:  The  Wiener  filter  problem 


given  Lj,  l^,  a and  6 determine  a causal  D so 

2 

as  to  minimize  E {|e|  }. 


FIGURE  4;  The  stochastic  estimation  problem 


given  a and  0 determine  a causal  D so  as  to 

i i2 

minimize  E {|D0  - a|  }. 


z ™ influence  over  the  future  output  of  the  plant  of  the  past 
input,  (P°ir) , and  the  perturbation,  (u);  z — PqFGB  (o>  + P°ir ) 


y “ output  of  the  plant 
u — control 


D — open  loop  regulator 
$ “ the  plant 


FIGURE  7:  The  optimal  regulator  problem: 

knowing  $ and  z determine  D so  as  to  minimize 

E (|»l*  * W2> 


FIGURE  9:  Kalman  optimal  regulator. 


t 


u — plant  input 

z =*  available  output  date 

w — perturbation 

A 

x “ estimated  state 

P°tt  ” past  input 

D — state  estimator 

x — plant  state 

$ — the  plant 

y "■  plant  input 

n “ measurement  noise 

FIGURE  10:  The  optimal  state  estimation  problem: 

knowing  $,  z and  u determine  a 

causal  D so  as  to 

- 2 

minimize  E {lx  - x|  }. 

z ” influence  over  the  future  output  of  the  plant  of  the 
past  input,  (P0it)»  the  perturbation,  (u>),  and  the  noise,  (n) 
z “ PqFGB  (<u  + P°ir)  + n- 

y “ output  of  the  plant  D — open  loop  regulator 

u - control  $ - the  plant 

FIGURE  13:  The  optimal  regulator  problem  in  the  noisy  case: 

knowing  $ and  z determine  D so  as  to  minimize" 

E {|u|2  + |y|2). 


FIGURE  15:  The  optimal  closed  loop  regulator  in  the  noisy  case. 


